1. Field of the Invention
The present invention relates to a method, system, and program for processing packets.
2. Description of the Related Art
In a network environment, a network adapter coupled to a host computer, such as an Ethernet card, Fibre Channel card, etc., will receive Input/Output (I/O) requests or responses to I/O requests initiated from the host. Often, the host computer operating system includes a device driver to communicate with the network adapter hardware to manage I/O requests to transmit over a network. Data packets received at the network adapter would be stored in an available allocated packet buffer in the host memory. The host computer further includes a transport protocol driver to process the packets received by the network adapter that are stored in the packet buffer, and access any I/O commands or data embedded in the packet. For instance, the transport protocol driver may implement the Transmission Control Protocol (TCP) and Internet Protocol (IP) to decode and access the payload data in the TCP/IP packets received at the network adapter.
The network adapter hardware maintains a ring of descriptors that point to buffers in the system memory to which the network adapter may write packets. A “descriptor” refers to a data structure that points to a buffer in memory that is used to store data, such as packets, where different buffer addresses may be assigned to the “descriptor”. The network adapter may use an on-board Direct Memory Access (DMA) engine to write packets to buffers in the host memory that are assigned to descriptors that are indicated as available. The device driver receives an interrupt from the network adapter indicating that packets were written to the buffers. The device driver will then wrap the packet in the buffer with information used by the operating system and make the buffer and wrapped packet available to the protocol driver, which may spawn a protocol stack thread to process. A different protocol stack thread may be spawned to process different packet buffers. When the spawned protocol stack thread completes processing the packet buffer, the protocol stack may return the buffer to the device driver by calling a return packet handler function, and the device driver may place the buffer into an available buffer pool. A return packet handler function is called to assign an available buffer to a descriptor and signal the network adapter hardware that the freed buffer and its corresponding descriptor are now available.
Because the protocol driver may spawn multiple threads to process packet buffers, multiple threads executing in one or more central processing units (CPUs) may be concurrently processing packet buffers written to by the network adapter. These spawned threads would also attempt to write to a software queue, such as the available buffer pool, and the hardware descriptors when completing processing of a packet buffer. In the prior art, a spin lock is used to serialize the attempts to update shared memory structures, such as the available buffer pool and the hardware descriptor in the network adapter, to indicate the availability of the reassigned descriptor. Thus, each spawned protocol stack thread must consume CPU resources to acquire and release the spin lock that is needed in order to update the hardware and software shared memory structures related to the descriptors. As the number of CPUs increase in a system, an increased amount of time and processing resources are consumed acquiring and releasing the spin lock.